feat(metrics): support Datadog metric origin metadata #18405
Conversation
✅ Deploy Preview for vrl-playground ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
✅ Deploy Preview for vector-project canceled.
|
github-actions
bot
added
domain: topology
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
/ci-run-regression |
Datadog ReportBranch report: ✅ |
|
/ci-run-regression |
Regression Detector ResultsRun ID: 157c491b-8fce-474b-8d3c-b2f0e4bb800d ExplanationA regression test is an integrated performance test for Because a target's optimization goal performance in each experiment will vary somewhat each time it is run, we can only estimate mean differences in optimization goal relative to the baseline target. We express these differences as a percentage change relative to the baseline target, denoted "Δ mean %". These estimates are made to a precision that balances accuracy and cost control. We represent this precision as a 90.00% confidence interval denoted "Δ mean % CI": there is a 90.00% chance that the true value of "Δ mean %" is in that interval. We decide whether a change in performance is a "regression" -- a change worth investigating further -- if both of the following two criteria are true:
The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values of "Δ mean %" mean that baseline is faster, whereas positive values of "Δ mean %" mean that comparison is faster. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed. No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%. Fine details of change detection per experiment.
|
| @@ -13,6 +13,19 @@ message CommonMetadata { | |||
| string api_key = 6; | |||
| } | |||
|
|
|||
| message Origin { | |||
There was a problem hiding this comment.
Note to reviewers: This was a change to prove this approach will work for Vector. Currently the official public proto file does not contain these additions. But hopefully will soon.
neuronull
added
source: datadog_agent
neuronull
added
the
domain: codecs
github-actions
bot
removed
the
domain: codecs
|
Note: this change is practically ready to merge, but we are awaiting confirmation that the Agent proto file changes are the same that have been made to this PR, which should ideally be known early next week. |
src/sinks/datadog/metrics/encoder.rs
Outdated
| }) | ||
| } | ||
| } | ||
|
|
||
| fn determine_origin_product_value() -> u32 { | ||
| option_env!("ORIGIN_PRODUCT") |
There was a problem hiding this comment.
Is this env var something we have named? I'm wondering if we should namespace it DD_ORIGIN_PRODUCT or VECTOR_ORIGIN_PRODUCT?
There was a problem hiding this comment.
Just noting that the Agent uses DD_ as the prefix for it's env variables which are particularly relevant when using the Agent in a container. Perhaps we should make this OPW_ to not cause confusion?
| // Some sources such as `kafka`, `nats`, `redis` for example, are only capable of receiving metrics | ||
| // with the `native` or `native_json` codec. In such cases we intentionally do not set the origin | ||
| // metadata here, because the true origin will have already been determined to be a pass-through. | ||
| fn source_type_to_service(source_type: &'static str) -> Option<u32> { |
There was a problem hiding this comment.
This is absolutely going to be forgotten about when we add new sources. I think we need a more robust design here. Could we perhaps add this number to DataType::Metric and thus force all metric sources to specify it?
There was a problem hiding this comment.
We did actually talk about this but the thinking was that we wanted to try to isolate the impact of this change to the datadog_metrics sink since that is the one that really needs it rather than leak it into source components. I do see what you are saying about the risk of forgetting to add it to a new metrics source. I wonder if we could also resolve that through testing?
There was a problem hiding this comment.
Are there scenarios here where it is Ok to return None? If there aren't, could we at least log a warning?
There was a problem hiding this comment.
I don't think a warning log would be an appropriate signal for this because this isn't really a user error or runtime error.
There was a problem hiding this comment.
And there are scenarios where we want to return None, such as when the source type is the datadog agent source.
There was a problem hiding this comment.
My initial proposal for this in the design, was to make the magic number forced at compile time in the source definitions. Sources that didn't need it would set it to none. But metric sources would be required to set it so adding new sources we would not overlook this.
I think @jszwedko was the primary voice of concern in the design discussion about spreading the logic of this feature out into the code base.
I think the current design would be a bit challenging to cover via testing (though I guess not impossible necessarily), because we would need to set something up to dynamically create a pipeline of [all metric sources] -> [dd metrics sink] , and validate that the non-agent sources that are metric sources, are setting the origin metadata correctly.
Otherwise, we run into the issue of having to remember to update unit tests (say if we tested this on the source side for the presence of source_type being set) , when adding new sources, which is just moving the remembering problem to the unit tests.
But I think at a minimum, if we proceed with the current approach, we could update the documentation and reviewer checklist we have for new sources, to call this out. The magic number will need to be added upstream as well so that adds another layer to adding new sources, esp for external contributors since we will have to take that part on. This technically could be done as a follow-up PR to the source getting merged into master.
There was a problem hiding this comment.
I don't think a warning log would be an appropriate signal for this because this isn't really a user error or runtime error.
I think it would make sense to have a log, but agree not at warn level. In the follow up PR I am drafting, I added a debug log to cover the case where we encounter a source_type that doesn't have a mapping here, which really is a bug on our part but not one we need to trouble users with.
Ofc, if we decide to change the design to enforce this through compilation, this case would not occur.
| } | ||
|
|
||
| fn generate_series_metadata( | ||
| maybe_pass_through: Option<&DatadogMetricOriginMetadata>, |
There was a problem hiding this comment.
Total nit, feel free to ignore. I'm not keen on the maybe_ naming. The maybe bit is apparent from the type and thus doesn't really add anything to the meaning.
There was a problem hiding this comment.
Yeah, I've always been iffy on these too. We have a few examples of MaybeXs for options in the code base, but it'd be good to develop some guidance here and add it to STYLE_GUIDE.md so we can converge on once style.
There was a problem hiding this comment.
I definitely modeled that after seeing other areas of the code base using that convention.
@StephenWakely just curious if you think that is blocking or if we could do it as a quick follow since we'd like to get this into the next Vector release. We could follow up on this question immediately though, once @neuronull is back the week after next. |
|
|
||
| // no metadata has been set upstream | ||
| } else { | ||
| maybe_source_type.and_then(|source_type| { |
There was a problem hiding this comment.
In reference to @StephenWakely's comment below, I think we should set the origin metadata if the source_type is not the datadog agent source but unknown. In that case, we would set the service to unknown as well. This would be slightly more robust than not setting the metadata at all in those scenarios in my opinion..
| // Some sources such as `kafka`, `nats`, `redis` for example, are only capable of receiving metrics | ||
| // with the `native` or `native_json` codec. In such cases we intentionally do not set the origin | ||
| // metadata here, because the true origin will have already been determined to be a pass-through. | ||
| fn source_type_to_service(source_type: &'static str) -> Option<u32> { |
There was a problem hiding this comment.
I don't think a warning log would be an appropriate signal for this because this isn't really a user error or runtime error.
| }) | ||
| } | ||
| } | ||
|
|
||
| fn determine_origin_product_value() -> u32 { |
There was a problem hiding this comment.
This is a good candidate for a once_cell::sync::Lazy.
Regression Detector ResultsRun ID: ce09b371-f0eb-40cd-a5ce-7f347b931b2f ExplanationA regression test is an integrated performance test for Because a target's optimization goal performance in each experiment will vary somewhat each time it is run, we can only estimate mean differences in optimization goal relative to the baseline target. We express these differences as a percentage change relative to the baseline target, denoted "Δ mean %". These estimates are made to a precision that balances accuracy and cost control. We represent this precision as a 90.00% confidence interval denoted "Δ mean % CI": there is a 90.00% chance that the true value of "Δ mean %" is in that interval. We decide whether a change in performance is a "regression" -- a change worth investigating further -- if both of the following two criteria are true:
The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values of "Δ mean %" mean that baseline is faster, whereas positive values of "Δ mean %" mean that comparison is faster. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed. No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%. Fine details of change detection per experiment.
|
# [2.0.0](answerbook/vector@v1.36.2...v2.0.0) (2024-01-09) ### Bug Fixes * **aws provider**: Don't unwap external_id (vectordotdev#18452) [fd0ccd5](answerbook/vector@fd0ccd5) - GitHub * **aws_s3 source**: Allow region to be optional (vectordotdev#18258) [23a1a2d](answerbook/vector@23a1a2d) - GitHub * **aws_s3 source**: Use the decoder to calculate type defs (vectordotdev#18274) [40f525c](answerbook/vector@40f525c) - GitHub * **ci**: add comment trigger filter for regression workflow concurrency group (vectordotdev#18408) [ff6e888](answerbook/vector@ff6e888) - GitHub * **ci**: don't continue on errors in unit-mac test (vectordotdev#18496) [712a210](answerbook/vector@712a210) - GitHub * **ci**: Drop docker-compose from bootstrap install (vectordotdev#18407) [41c5567](answerbook/vector@41c5567) - GitHub * **ci**: Unlink python before `brew install` (vectordotdev#18402) [e39d9b3](answerbook/vector@e39d9b3) - GitHub * **config**: fix concurrency default & docs (vectordotdev#18651) [b35527d](answerbook/vector@b35527d) - GitHub * **config**: Only try default paths if the path is not specified (vectordotdev#18681) [b050a65](answerbook/vector@b050a65) - Jesse Szwedko * **core**: don't show warning about max allocation groups if tracing not enabled (vectordotdev#18589) [3afda3c](answerbook/vector@3afda3c) - GitHub * **datadog_metrics sink**: improve aggregation performance (vectordotdev#18759) [4f97d48](answerbook/vector@4f97d48) - Jesse Szwedko * **debian platform**: Re-add `conf-files` directive for `cargo-deb` (vectordotdev#18726) [aa39f5e](answerbook/vector@aa39f5e) - Jesse Szwedko * **debian platform**: Remove `conf-files` directive for `cargo-deb` (vectordotdev#18455) [40ef7c4](answerbook/vector@40ef7c4) - GitHub * default to nullbyte delimiter for GELF vectordotdev#18008 (vectordotdev#18419) [a112704](answerbook/vector@a112704) - GitHub * **deps, security**: temporarily ignore `ed25519-dalek` security vulnerability (vectordotdev#18245) [1b90398](answerbook/vector@1b90398) - GitHub * **deps**: fix [dev-dependencies] for some libs (vectordotdev#18328) [8086e19](answerbook/vector@8086e19) - GitHub * **deps**: load default and legacy openssl providers (vectordotdev#18276) [fc17fba](answerbook/vector@fc17fba) - GitHub * **distribution**: Use yaml instead of toml file (vectordotdev#18606) [24701cf](answerbook/vector@24701cf) - GitHub * **dnstap source**: support DNSSEC RRSIG record data (vectordotdev#18878) [3eaad37](answerbook/vector@3eaad37) - Jesse Szwedko * **docs**: add the 'http_client_requests_sent_total' (vectordotdev#18299) [69e8383](answerbook/vector@69e8383) - GitHub * **docs**: remove '---\n' prefix from toYaml config example generator (vectordotdev#18502) [8d07e18](answerbook/vector@8d07e18) - GitHub * **elasticsearch**: Ignore `pipeline` argument if it is an empty string (vectordotdev#18248) [d9dbed8](answerbook/vector@d9dbed8) - GitHub * **external docs**: Document intentional label on component_discarded_events_total (vectordotdev#18622) [fd58af9](answerbook/vector@fd58af9) - GitHub * **gcp service**: retry on unauthorized (vectordotdev#18586) [75d03b3](answerbook/vector@75d03b3) - GitHub * **json codec**: Fix deserializing non-object values with the `Vector` namespace (vectordotdev#18379) [f15144b](answerbook/vector@f15144b) - GitHub * **kafka sink**: Make KafkaService return `Poll::Pending` when producer queue is full (vectordotdev#18770) [fa09de3](answerbook/vector@fa09de3) - Jesse Szwedko * **kafka sink**: performance improvements and fix memory leak (vectordotdev#18634) [3c662f3](answerbook/vector@3c662f3) - GitHub * **kubernetes_logs source**: Fix events being empty when log namespacing is enabled (vectordotdev#18244) [8918c66](answerbook/vector@8918c66) - GitHub * **new_relic sink**: Multiple fixes related to metrics (vectordotdev#18151) [953e305](answerbook/vector@953e305) - GitHub * **observability**: add all events that are being encoded (vectordotdev#18289) [93baba2](answerbook/vector@93baba2) - GitHub * **observability**: don't increment component_errors_total for `HttpClient` warning (vectordotdev#18505) [a544e6e](answerbook/vector@a544e6e) - GitHub * **opentelemetry source**: Remove the 4MB default for gRPC request decoding (vectordotdev#18306) [a6262cd](answerbook/vector@a6262cd) - GitHub * **prometheus_remote_write source, prometheus_scrape source**: Fix feature check (vectordotdev#18440) [9c1abd6](answerbook/vector@9c1abd6) - GitHub * **releasing**: Update example YAML config data_dir (vectordotdev#18896) [ac86a8a](answerbook/vector@ac86a8a) - Jesse Szwedko * **remap transform**: log namespace should be used when splitting events from arrays (vectordotdev#18372) [15a63b4](answerbook/vector@15a63b4) - GitHub * **sample transform**: Use metadata when log namespacing is enabled (vectordotdev#18259) [69b4c1c](answerbook/vector@69b4c1c) - GitHub * **sinks**: resolve memory leak by always setting a request builder concurrency limit (vectordotdev#18637) [6ced6ca](answerbook/vector@6ced6ca) - GitHub * skip encoding empty sketches (vectordotdev#18530) [ae0aa11](answerbook/vector@ae0aa11) - GitHub * socket tcp port typedef (vectordotdev#18180) [b0c89ab](answerbook/vector@b0c89ab) - GitHub * **syslog source, docs**: Fix docs for `host` field for syslog source (vectordotdev#18453) [0ef902d](answerbook/vector@0ef902d) - GitHub * **tests**: fix tests for the generate command (vectordotdev#18383) [2a4235c](answerbook/vector@2a4235c) - GitHub * use `rstest` in `generate` command tests (vs wrong usage of `proptest`) (vectordotdev#18365) [4359c9a](answerbook/vector@4359c9a) - GitHub * **website**: Fix installer list for MacOS (vectordotdev#18364) [13eec06](answerbook/vector@13eec06) - GitHub ### Chores * Add SHA256 checksums file to GH releases (vectordotdev#18701) [3411642](answerbook/vector@3411642) - Jesse Szwedko * **appsignal sink**: Refactor to use StreamSink (vectordotdev#18209) [c495939](answerbook/vector@c495939) - GitHub * **aws provider, external_docs**: Update the AWS authentication documentation (vectordotdev#18492) [cd8c5fe](answerbook/vector@cd8c5fe) - GitHub * **ci**: Add a test to assert conf files aren't overwritten (vectordotdev#18728) [ab272f6](answerbook/vector@ab272f6) - Jesse Szwedko * **ci**: Bump actions/checkout from 3 to 4 (vectordotdev#18476) [59dfd67](answerbook/vector@59dfd67) - GitHub * **ci**: Bump aws-actions/configure-aws-credentials from 2.2.0 to 3.0.1 (vectordotdev#18386) [f33aff1](answerbook/vector@f33aff1) - GitHub * **ci**: Bump aws-actions/configure-aws-credentials from 3.0.1 to 3.0.2 (vectordotdev#18511) [737d2f1](answerbook/vector@737d2f1) - GitHub * **ci**: Bump aws-actions/configure-aws-credentials from 3.0.2 to 4.0.0 (vectordotdev#18544) [3018864](answerbook/vector@3018864) - GitHub * **ci**: Bump docker/build-push-action from 4.1.1 to 4.2.1 (vectordotdev#18512) [f19d166](answerbook/vector@f19d166) - GitHub * **ci**: Bump docker/build-push-action from 4.2.1 to 5.0.0 (vectordotdev#18546) [c30a4b2](answerbook/vector@c30a4b2) - GitHub * **ci**: Bump docker/login-action from 2 to 3 (vectordotdev#18556) [d976c6e](answerbook/vector@d976c6e) - GitHub * **ci**: Bump docker/metadata-action from 4.6.0 to 5.0.0 (vectordotdev#18543) [8aae235](answerbook/vector@8aae235) - GitHub * **ci**: Bump docker/setup-buildx-action from 2.10.0 to 3.0.0 (vectordotdev#18545) [899e3c0](answerbook/vector@899e3c0) - GitHub * **ci**: Bump docker/setup-buildx-action from 2.9.1 to 2.10.0 (vectordotdev#18406) [dd8a0ef](answerbook/vector@dd8a0ef) - GitHub * **ci**: Bump docker/setup-qemu-action from 2.2.0 to 3.0.0 (vectordotdev#18547) [dcda1de](answerbook/vector@dcda1de) - GitHub * **ci**: Bump MacOS unit test runners to 13 (vectordotdev#18823) [c499427](answerbook/vector@c499427) - Jesse Szwedko * **ci**: Bump myrotvorets/set-commit-status-action from 1.1.7 to 2.0.0 (vectordotdev#18510) [8d5003a](answerbook/vector@8d5003a) - GitHub * **ci**: Bump tibdex/github-app-token from 1.8.0 to 1.8.2 (vectordotdev#18454) [4e4ece6](answerbook/vector@4e4ece6) - GitHub * **ci**: Bump tibdex/github-app-token from 1.8.2 to 2.0.0 (vectordotdev#18528) [7295f22](answerbook/vector@7295f22) - GitHub * **ci**: Bump tibdex/github-app-token from 2.0.0 to 2.1.0 (vectordotdev#18608) [addc46e](answerbook/vector@addc46e) - GitHub * **ci**: Fix cookie banner style issues (vectordotdev#18745) [792a1b5](answerbook/vector@792a1b5) - Jesse Szwedko * **ci**: group azure and prost crates for dependabot (vectordotdev#18525) [7b6ad62](answerbook/vector@7b6ad62) - GitHub * **ci**: Re-add docker-compose installation (vectordotdev#18415) [c1ed017](answerbook/vector@c1ed017) - GitHub * **ci**: remove kinetic as it's no longer supported (vectordotdev#18540) [c74a469](answerbook/vector@c74a469) - GitHub * **ci**: revert bump actions/checkout from 3 to 4 (vectordotdev#18490) [3fd6486](answerbook/vector@3fd6486) - GitHub * **config, docs**: Replace 'vector.toml' with 'vector.yaml' (vectordotdev#18388) [9d29563](answerbook/vector@9d29563) - GitHub * **datadog_logs sink**: Use `endpoint` config setting consistent with the other datadog_ sinks. (vectordotdev#18497) [1ac19dd](answerbook/vector@1ac19dd) - GitHub * **deps**: Bump `nkeys` to 0.3.2 (vectordotdev#18264) [fc533a1](answerbook/vector@fc533a1) - GitHub * **deps**: Bump anyhow from 1.0.72 to 1.0.74 (vectordotdev#18255) [a77b652](answerbook/vector@a77b652) - GitHub * **deps**: Bump anyhow from 1.0.74 to 1.0.75 (vectordotdev#18284) [fb5f099](answerbook/vector@fb5f099) - GitHub * **deps**: Bump async-compression from 0.4.1 to 0.4.2 (vectordotdev#18417) [8639655](answerbook/vector@8639655) - GitHub * **deps**: Bump async-compression from 0.4.2 to 0.4.3 (vectordotdev#18539) [3e3c7ad](answerbook/vector@3e3c7ad) - GitHub * **deps**: Bump async-nats from 0.31.0 to 0.32.0 (vectordotdev#18640) [7ecb06a](answerbook/vector@7ecb06a) - GitHub * **deps**: Bump async-recursion from 1.0.4 to 1.0.5 (vectordotdev#18466) [8b017b6](answerbook/vector@8b017b6) - GitHub * **deps**: Bump azure_core from 0.13.0 to 0.14.0 (vectordotdev#18361) [0bbb152](answerbook/vector@0bbb152) - GitHub * **deps**: Bump base64 from 0.21.2 to 0.21.3 (vectordotdev#18398) [5dca377](answerbook/vector@5dca377) - GitHub * **deps**: Bump base64 from 0.21.3 to 0.21.4 (vectordotdev#18522) [44a87dc](answerbook/vector@44a87dc) - GitHub * **deps**: Bump bollard from 0.14.0 to 0.15.0 (vectordotdev#18581) [996372f](answerbook/vector@996372f) - GitHub * **deps**: Bump bstr from 1.6.0 to 1.6.1 (vectordotdev#18422) [fa6f2cd](answerbook/vector@fa6f2cd) - GitHub * **deps**: Bump bstr from 1.6.1 to 1.6.2 (vectordotdev#18433) [8b6a307](answerbook/vector@8b6a307) - GitHub * **deps**: Bump bytes from 1.4.0 to 1.5.0 (vectordotdev#18508) [bfdb5b0](answerbook/vector@bfdb5b0) - GitHub * **deps**: Bump bytesize from 1.2.0 to 1.3.0 (vectordotdev#18367) [836a31e](answerbook/vector@836a31e) - GitHub * **deps**: Bump cached from 0.44.0 to 0.45.0 (vectordotdev#18478) [325fbea](answerbook/vector@325fbea) - GitHub * **deps**: Bump cached from 0.45.0 to 0.45.1 (vectordotdev#18485) [f8981e1](answerbook/vector@f8981e1) - GitHub * **deps**: Bump cargo_toml from 0.15.3 to 0.16.0 (vectordotdev#18571) [0e60001](answerbook/vector@0e60001) - GitHub * **deps**: Bump cargo_toml from 0.16.0 to 0.16.1 (vectordotdev#18605) [c47e65f](answerbook/vector@c47e65f) - GitHub * **deps**: Bump cargo_toml from 0.16.1 to 0.16.2 (vectordotdev#18619) [fa526a8](answerbook/vector@fa526a8) - GitHub * **deps**: Bump cargo_toml from 0.16.2 to 0.16.3 (vectordotdev#18674) [a4cd8b7](answerbook/vector@a4cd8b7) - GitHub * **deps**: Bump chrono to 0.4.30 (vectordotdev#18527) [a6305de](answerbook/vector@a6305de) - GitHub * **deps**: Bump cidr-utils from 0.5.10 to 0.5.11 (vectordotdev#18516) [f5cd27a](answerbook/vector@f5cd27a) - GitHub * **deps**: Bump clap from 4.3.21 to 4.3.22 (vectordotdev#18293) [47f25d3](answerbook/vector@47f25d3) - GitHub * **deps**: Bump clap from 4.3.22 to 4.3.23 (vectordotdev#18311) [05765d8](answerbook/vector@05765d8) - GitHub * **deps**: Bump clap from 4.3.23 to 4.3.24 (vectordotdev#18362) [5f4a6d8](answerbook/vector@5f4a6d8) - GitHub * **deps**: Bump clap from 4.3.24 to 4.4.1 (vectordotdev#18411) [d3b9eda](answerbook/vector@d3b9eda) - GitHub * **deps**: Bump clap from 4.4.1 to 4.4.2 (vectordotdev#18447) [6998518](answerbook/vector@6998518) - GitHub * **deps**: Bump clap from 4.4.2 to 4.4.3 (vectordotdev#18550) [9e8407e](answerbook/vector@9e8407e) - GitHub * **deps**: Bump clap_complete from 4.3.2 to 4.4.0 (vectordotdev#18374) [76ffdab](answerbook/vector@76ffdab) - GitHub * **deps**: Bump clap_complete from 4.4.0 to 4.4.1 (vectordotdev#18509) [af444ea](answerbook/vector@af444ea) - GitHub * **deps**: Bump clap_complete from 4.4.1 to 4.4.2 (vectordotdev#18673) [3edafef](answerbook/vector@3edafef) - GitHub * **deps**: Bump crossterm from 0.26.1 to 0.27.0 (vectordotdev#18168) [d9c75bd](answerbook/vector@d9c75bd) - GitHub * **deps**: Bump dashmap from 5.5.0 to 5.5.1 (vectordotdev#18338) [754bee0](answerbook/vector@754bee0) - GitHub * **deps**: Bump dashmap from 5.5.1 to 5.5.3 (vectordotdev#18427) [adf134b](answerbook/vector@adf134b) - GitHub * **deps**: Bump dyn-clone from 1.0.12 to 1.0.13 (vectordotdev#18285) [1fbbdcb](answerbook/vector@1fbbdcb) - GitHub * **deps**: Bump dyn-clone from 1.0.13 to 1.0.14 (vectordotdev#18607) [d5f4caa](answerbook/vector@d5f4caa) - GitHub * **deps**: Bump encoding_rs from 0.8.32 to 0.8.33 (vectordotdev#18360) [2493288](answerbook/vector@2493288) - GitHub * **deps**: Bump enumflags2 from 0.7.7 to 0.7.8 (vectordotdev#18560) [0f90c39](answerbook/vector@0f90c39) - GitHub * **deps**: Bump flate2 from 1.0.26 to 1.0.27 (vectordotdev#18254) [3a6af99](answerbook/vector@3a6af99) - GitHub * **deps**: Bump h2 from 0.3.20 to 0.3.21 (vectordotdev#18330) [d41f9f6](answerbook/vector@d41f9f6) - GitHub * **deps**: Bump headers from 0.3.8 to 0.3.9 (vectordotdev#18448) [749594c](answerbook/vector@749594c) - GitHub * **deps**: Bump http-serde from 1.1.2 to 1.1.3 (vectordotdev#18310) [31ec4b3](answerbook/vector@31ec4b3) - GitHub * **deps**: Bump indicatif from 0.17.6 to 0.17.7 (vectordotdev#18647) [6bab5be](answerbook/vector@6bab5be) - GitHub * **deps**: Bump indoc from 2.0.3 to 2.0.4 (vectordotdev#18582) [b3f76b5](answerbook/vector@b3f76b5) - GitHub * **deps**: Bump inventory from 0.3.11 to 0.3.12 (vectordotdev#18429) [9f95026](answerbook/vector@9f95026) - GitHub * **deps**: Bump libc from 0.2.147 to 0.2.148 (vectordotdev#18563) [59e22fc](answerbook/vector@59e22fc) - GitHub * **deps**: Bump lru from 0.11.0 to 0.11.1 (vectordotdev#18484) [7ec5b97](answerbook/vector@7ec5b97) - GitHub * **deps**: Bump md-5 from 0.10.5 to 0.10.6 (vectordotdev#18648) [e8d946f](answerbook/vector@e8d946f) - GitHub * **deps**: Bump memchr from 2.5.0 to 2.6.0 (vectordotdev#18410) [7bc1942](answerbook/vector@7bc1942) - GitHub * **deps**: Bump memchr from 2.6.0 to 2.6.1 (vectordotdev#18421) [3f4603c](answerbook/vector@3f4603c) - GitHub * **deps**: Bump memchr from 2.6.1 to 2.6.2 (vectordotdev#18434) [5e3eaa9](answerbook/vector@5e3eaa9) - GitHub * **deps**: Bump memchr from 2.6.2 to 2.6.3 (vectordotdev#18470) [69a1ca0](answerbook/vector@69a1ca0) - GitHub * **deps**: Bump mlua from 0.8.9 to 0.8.10 (vectordotdev#18292) [03fe2fe](answerbook/vector@03fe2fe) - GitHub * **deps**: Bump mongodb from 2.6.0 to 2.6.1 (vectordotdev#18268) [28de959](answerbook/vector@28de959) - GitHub * **deps**: Bump MSRV to 1.70.0 (vectordotdev#18394) [fd21b19](answerbook/vector@fd21b19) - GitHub * **deps**: Bump no-proxy from 0.3.3 to 0.3.4 (vectordotdev#18277) [7a1c49c](answerbook/vector@7a1c49c) - GitHub * **deps**: Bump notify from 6.0.1 to 6.1.0 (vectordotdev#18317) [e963766](answerbook/vector@e963766) - GitHub * **deps**: Bump notify from 6.1.0 to 6.1.1 (vectordotdev#18332) [ae5de9c](answerbook/vector@ae5de9c) - GitHub * **deps**: Bump num_enum from 0.6.1 to 0.7.0 (vectordotdev#18238) [771f476](answerbook/vector@771f476) - GitHub * **deps**: Bump openssl from 0.10.56 to 0.10.57 (vectordotdev#18400) [15792f6](answerbook/vector@15792f6) - GitHub * **deps**: Bump ordered-float from 3.7.0 to 3.8.0 (vectordotdev#18302) [833ac19](answerbook/vector@833ac19) - GitHub * **deps**: Bump ordered-float from 3.8.0 to 3.9.0 (vectordotdev#18307) [ca30b6d](answerbook/vector@ca30b6d) - GitHub * **deps**: Bump ordered-float from 3.9.0 to 3.9.1 (vectordotdev#18350) [ab41edc](answerbook/vector@ab41edc) - GitHub * **deps**: Bump proc-macro2 from 1.0.66 to 1.0.67 (vectordotdev#18561) [6f091e1](answerbook/vector@6f091e1) - GitHub * **deps**: Bump prost from 0.11.9 to 0.12.0 (vectordotdev#18460) [4fba377](answerbook/vector@4fba377) - GitHub * **deps**: Bump prost-reflect from 0.11.4 to 0.11.5 (vectordotdev#18426) [221e0a1](answerbook/vector@221e0a1) - GitHub * **deps**: Bump quote from 1.0.32 to 1.0.33 (vectordotdev#18283) [f2a6887](answerbook/vector@f2a6887) - GitHub * **deps**: Bump ratatui from 0.22.0 to 0.23.0 (vectordotdev#18412) [82be883](answerbook/vector@82be883) - GitHub * **deps**: Bump rdkafka from 0.33.2 to 0.34.0 (vectordotdev#18393) [016890e](answerbook/vector@016890e) - GitHub * **deps**: Bump redis from 0.23.2 to 0.23.3 (vectordotdev#18464) [2e2692e](answerbook/vector@2e2692e) - GitHub * **deps**: Bump regex from 1.9.3 to 1.9.4 (vectordotdev#18399) [21f7679](answerbook/vector@21f7679) - GitHub * **deps**: Bump regex from 1.9.4 to 1.9.5 (vectordotdev#18472) [1aeb42e](answerbook/vector@1aeb42e) - GitHub * **deps**: Bump reqwest from 0.11.18 to 0.11.19 (vectordotdev#18329) [9b4625c](answerbook/vector@9b4625c) - GitHub * **deps**: Bump reqwest from 0.11.19 to 0.11.20 (vectordotdev#18366) [2aaea89](answerbook/vector@2aaea89) - GitHub * **deps**: Bump rmpv from 1.0.0 to 1.0.1 (vectordotdev#18049) [0cda906](answerbook/vector@0cda906) - GitHub * **deps**: Bump Rust to 1.72.1 (vectordotdev#18638) [466ef84](answerbook/vector@466ef84) - GitHub * **deps**: Bump serde from 1.0.183 to 1.0.185 (vectordotdev#18319) [ea2d576](answerbook/vector@ea2d576) - GitHub * **deps**: Bump serde from 1.0.185 to 1.0.186 (vectordotdev#18370) [83ec3cf](answerbook/vector@83ec3cf) - GitHub * **deps**: Bump serde from 1.0.186 to 1.0.188 (vectordotdev#18395) [4a2805d](answerbook/vector@4a2805d) - GitHub * **deps**: Bump serde_derive_internals from 0.28.0 to 0.29.0 (vectordotdev#18499) [f7c3c96](answerbook/vector@f7c3c96) - GitHub * **deps**: Bump serde_json from 1.0.104 to 1.0.105 (vectordotdev#18267) [d05bc3e](answerbook/vector@d05bc3e) - GitHub * **deps**: Bump serde_json from 1.0.105 to 1.0.106 (vectordotdev#18523) [2687ed1](answerbook/vector@2687ed1) - GitHub * **deps**: Bump serde_json from 1.0.106 to 1.0.107 (vectordotdev#18562) [3c3e251](answerbook/vector@3c3e251) - GitHub * **deps**: Bump serde_with from 3.2.0 to 3.3.0 (vectordotdev#18315) [54d48d7](answerbook/vector@54d48d7) - GitHub * **deps**: Bump serde-wasm-bindgen from 0.5.0 to 0.6.0 (vectordotdev#18565) [95297b2](answerbook/vector@95297b2) - GitHub * **deps**: Bump similar-asserts from 1.4.2 to 1.5.0 (vectordotdev#18318) [b3a1d53](answerbook/vector@b3a1d53) - GitHub * **deps**: Bump smallvec from 1.11.0 to 1.11.1 (vectordotdev#18620) [feca4c8](answerbook/vector@feca4c8) - GitHub * **deps**: Bump socket2 from 0.5.3 to 0.5.4 (vectordotdev#18531) [8eb5256](answerbook/vector@8eb5256) - GitHub * **deps**: Bump syn from 2.0.28 to 2.0.29 (vectordotdev#18282) [d10e373](answerbook/vector@d10e373) - GitHub * **deps**: Bump syn from 2.0.29 to 2.0.31 (vectordotdev#18471) [f2b46d6](answerbook/vector@f2b46d6) - GitHub * **deps**: Bump syn from 2.0.31 to 2.0.32 (vectordotdev#18524) [f30537e](answerbook/vector@f30537e) - GitHub * **deps**: Bump syn from 2.0.32 to 2.0.33 (vectordotdev#18559) [f750cf7](answerbook/vector@f750cf7) - GitHub * **deps**: Bump syn from 2.0.33 to 2.0.37 (vectordotdev#18580) [e80d7b7](answerbook/vector@e80d7b7) - GitHub * **deps**: Bump the azure group with 4 updates (vectordotdev#18529) [3d7199e](answerbook/vector@3d7199e) - GitHub * **deps**: Bump the prost group with 3 updates (vectordotdev#18579) [16edc22](answerbook/vector@16edc22) - GitHub * **deps**: Bump thiserror from 1.0.44 to 1.0.46 (vectordotdev#18253) [2a8d974](answerbook/vector@2a8d974) - GitHub * **deps**: Bump thiserror from 1.0.46 to 1.0.47 (vectordotdev#18286) [378926d](answerbook/vector@378926d) - GitHub * **deps**: Bump thiserror from 1.0.47 to 1.0.48 (vectordotdev#18473) [7ad6313](answerbook/vector@7ad6313) - GitHub * **deps**: Bump to Rust 1.72.0 (vectordotdev#18389) [7849d80](answerbook/vector@7849d80) - GitHub * **deps**: Bump tokio from 1.30.0 to 1.32.0 (vectordotdev#18279) [47051a5](answerbook/vector@47051a5) - GitHub * **deps**: Bump tokio-postgres from 0.7.7 to 0.7.9 (vectordotdev#18316) [a7800f7](answerbook/vector@a7800f7) - GitHub * **deps**: Bump tokio-postgres from 0.7.9 to 0.7.10 (vectordotdev#18391) [725b9bd](answerbook/vector@725b9bd) - GitHub * **deps**: Bump tokio-test from 0.4.2 to 0.4.3 (vectordotdev#18357) [6716959](answerbook/vector@6716959) - GitHub * **deps**: Bump tokio-tungstenite from 0.20.0 to 0.20.1 (vectordotdev#18661) [42fea39](answerbook/vector@42fea39) - Jesse Szwedko * **deps**: Bump toml from 0.7.6 to 0.7.7 (vectordotdev#18507) [688e2d9](answerbook/vector@688e2d9) - GitHub * **deps**: Bump toml from 0.7.7 to 0.7.8 (vectordotdev#18520) [0476bb5](answerbook/vector@0476bb5) - GitHub * **deps**: Bump toml from 0.7.8 to 0.8.0 (vectordotdev#18549) [0382bc5](answerbook/vector@0382bc5) - GitHub * **deps**: Bump tonic from 0.10.0 to 0.10.1 (vectordotdev#18639) [9523987](answerbook/vector@9523987) - GitHub * **deps**: Bump tonic-build from 0.10.0 to 0.10.1 (vectordotdev#18641) [ec9efb5](answerbook/vector@ec9efb5) - GitHub * **deps**: Bump tower-http from 0.4.3 to 0.4.4 (vectordotdev#18461) [d3d8a4c](answerbook/vector@d3d8a4c) - GitHub * **deps**: Bump trust-dns-proto from 0.22.0 to 0.23.0 (vectordotdev#18349) [a252eda](answerbook/vector@a252eda) - GitHub * **deps**: Bump typetag from 0.2.12 to 0.2.13 (vectordotdev#18287) [704cbfe](answerbook/vector@704cbfe) - GitHub * **deps**: Bump url from 2.4.0 to 2.4.1 (vectordotdev#18414) [314a37b](answerbook/vector@314a37b) - GitHub * **deps**: Bump warp from 0.3.5 to 0.3.6 (vectordotdev#18704) [0a5d29e](answerbook/vector@0a5d29e) - Jesse Szwedko * **deps**: Bump webpki 0.22.1 -> 0.22.4 [d9c4f2e](answerbook/vector@d9c4f2e) - Jesse Szwedko * **deps**: Bump webpki from 0.22.0 to 0.22.1 (vectordotdev#18494) [9859b9e](answerbook/vector@9859b9e) - GitHub * **deps**: Bump webpki from 0.22.1 to 0.22.2 (vectordotdev#18744) [2cbc16c](answerbook/vector@2cbc16c) - Jesse Szwedko * **deps**: Drop patch to use custom `chrono` repo (vectordotdev#18567) [6c34cd5](answerbook/vector@6c34cd5) - GitHub * **deps**: Drop use of `once_cell::{sync,unsync}::OnceCell` (vectordotdev#17621) [f2cd59a](answerbook/vector@f2cd59a) - GitHub * **deps**: Fix issue with `cargo` refetching refs on every run (vectordotdev#18331) [6397edb](answerbook/vector@6397edb) - GitHub * **deps**: group tonic crates in dependabot (vectordotdev#18645) [32be0d3](answerbook/vector@32be0d3) - GitHub * **deps**: Remove openssl-sys patch (vectordotdev#18495) [8ec87eb](answerbook/vector@8ec87eb) - GitHub * **deps**: Update fork of rust-openssl (vectordotdev#18404) [f8d073e](answerbook/vector@f8d073e) - GitHub * **deps**: Update VRL to 0.7.0 (vectordotdev#18672) [70e8b5f](answerbook/vector@70e8b5f) - GitHub * **dev**: Add CODEOWNERS for documentation updates (vectordotdev#18628) [918beac](answerbook/vector@918beac) - GitHub * **dev**: Add DEPRECATIONS.md file to track deprecations (vectordotdev#18613) [d8f36e4](answerbook/vector@d8f36e4) - GitHub * **distribution**: Fix PATH modification to allow for spaces (vectordotdev#18294) [aed8224](answerbook/vector@aed8224) - GitHub * **docs**: Add highlight announcing YAML as the new default config … (vectordotdev#18435) [c150b14](answerbook/vector@c150b14) - GitHub * **docs**: add more comparison examples (vectordotdev#18333) [2f458f6](answerbook/vector@2f458f6) - GitHub * **docs**: Change the default configuration tab and add comments to… (vectordotdev#18420) [ed1dedf](answerbook/vector@ed1dedf) - GitHub * **docs**: Convert a few more configs to YAML (vectordotdev#18632) [f2d60cb](answerbook/vector@f2d60cb) - GitHub * **docs**: discuss disk throughput configurations in sizing guidance (vectordotdev#18566) [5a52b61](answerbook/vector@5a52b61) - GitHub * **docs**: Emphasize the "may" bit of the backpressure docs (vectordotdev#18457) [1647964](answerbook/vector@1647964) - GitHub * **docs**: make YAML appear first in the example configurations (vectordotdev#18325) [85dd439](answerbook/vector@85dd439) - GitHub * **exec source**: remove obsolete codec file and import (vectordotdev#18257) [adbc06f](answerbook/vector@adbc06f) - GitHub * **exec source**: split tests into own module file (vectordotdev#18301) [ac90069](answerbook/vector@ac90069) - GitHub * **external docs**: Remove or replace mentions of vector in functions doc (vectordotdev#18679) [3910677](answerbook/vector@3910677) - Jesse Szwedko * feature gate aws-core features (vectordotdev#18482) [e19243f](answerbook/vector@e19243f) - GitHub * **gcp_stackdriver_logs sink**: refactor to new style (vectordotdev#18335) [0d8ab26](answerbook/vector@0d8ab26) - GitHub * **honeycomb sink**: refactor to new style (vectordotdev#18280) [e104dc4](answerbook/vector@e104dc4) - GitHub * **http sink**: refactor to new style (vectordotdev#18200) [d3a6235](answerbook/vector@d3a6235) - GitHub * Improve checksum script (vectordotdev#18487) [8981cba](answerbook/vector@8981cba) - GitHub * **internal docs**: Add workspace to new rust docs deployment (vectordotdev#18616) [180647b](answerbook/vector@180647b) - GitHub * **kubernetes**: Regenerate manifests from new Helm chart version (vectordotdev#18629) [99de28b](answerbook/vector@99de28b) - GitHub * **log_to_metric transform**: Add Cargo feature for the transform (vectordotdev#18337) [d044084](answerbook/vector@d044084) - GitHub * **log_to_metric transform**: add missing Cargo feature (vectordotdev#18308) [69621bd](answerbook/vector@69621bd) - GitHub * **log_to_metric transform**: Revert missing Cargo feature `transforms-log_to_metric` (vectordotdev#18327) [e6ec664](answerbook/vector@e6ec664) - GitHub * make pin-project a workspace dependency (vectordotdev#18176) [b755a46](answerbook/vector@b755a46) - GitHub * **nats sink**: Refactor to use StreamSink components (vectordotdev#18243) [294c1dd](answerbook/vector@294c1dd) - GitHub * **prometheus_remote_write source, prometheus_scrape source**: Split compilation features (vectordotdev#18431) [c07c99d](answerbook/vector@c07c99d) - GitHub * **redis sink**: Refactor to use StreamSink (vectordotdev#18220) [1e7e99c](answerbook/vector@1e7e99c) - GitHub * **releasing**: Add 0.32.0 highlight for legacy OpenSSL provider deprecation (vectordotdev#18263) [62bd340](answerbook/vector@62bd340) - GitHub * **releasing**: Add known issue for 0.33.0 debian packaging regression (vectordotdev#18727) [eae7b82](answerbook/vector@eae7b82) - Jesse Szwedko * **releasing**: Add known issues for v0.32.0 (vectordotdev#18298) [d09b2af](answerbook/vector@d09b2af) - GitHub * **releasing**: Add note about protobuf codec addition for 0.32.0 release (vectordotdev#18275) [87850ee](answerbook/vector@87850ee) - GitHub * **releasing**: Bump k8s manifests to v0.24.1 of the chart (vectordotdev#18334) [a1d05e4](answerbook/vector@a1d05e4) - GitHub * **releasing**: Bump Vector to 0.33.0 (vectordotdev#18250) [ef51e7a](answerbook/vector@ef51e7a) - GitHub * **releasing**: Prepare v0.32.0 release [50685f9](answerbook/vector@50685f9) - Jesse Szwedko * **releasing**: Prepare v0.32.1 release [bb42d52](answerbook/vector@bb42d52) - Jesse Szwedko * **releasing**: Prepare v0.32.2 release [d833296](answerbook/vector@d833296) - Jesse Szwedko * **releasing**: Prepare v0.33.0 release [89605fb](answerbook/vector@89605fb) - Jesse Szwedko * **releasing**: Prepare v0.33.1 release [3cc27b9](answerbook/vector@3cc27b9) - Jesse Szwedko * **releasing**: Regenerate k8s manifests for v0.24.0 of the chart (vectordotdev#18251) [d155a23](answerbook/vector@d155a23) - GitHub * **releasing**: Use large pages for better OS compatibility (vectordotdev#18481) [e23941c](answerbook/vector@e23941c) - GitHub * **rust-doc**: Add new Make command for CI builds (vectordotdev#18444) [4127b2c](answerbook/vector@4127b2c) - GitHub * **rust-doc**: Update amplify build to include workspace flag (vectordotdev#18630) [f01354e](answerbook/vector@f01354e) - GitHub * **security**: Make the warning for the deprecated OpenSSL provider more verbose (vectordotdev#18278) [7952bfb](answerbook/vector@7952bfb) - GitHub * tidy `encode_input` function (vectordotdev#18300) [2a45722](answerbook/vector@2a45722) - GitHub * update `rustls-webpki` due to security advisory (vectordotdev#18344) [b982a74](answerbook/vector@b982a74) - GitHub * Use `Ipv#Addr` constants (vectordotdev#17627) [7a6365e](answerbook/vector@7a6365e) - GitHub * **websites**: Add amplify build spec files to appropriate directories (vectordotdev#18668) [7cbb758](answerbook/vector@7cbb758) - GitHub * **website**: Set download page dropdown to latest version (vectordotdev#18758) [4b72f7e](answerbook/vector@4b72f7e) - Jesse Szwedko * **website**: Update chat.vector.dev redirect (vectordotdev#18635) [76efb4b](answerbook/vector@76efb4b) - GitHub ### Features * Add checksums for artifacts (vectordotdev#18483) [f11eeb3](answerbook/vector@f11eeb3) - GitHub * add convert config command (vectordotdev#18378) [9cb0ca3](answerbook/vector@9cb0ca3) - GitHub * Add git sha to the VRL playground header (vectordotdev#18500) [567de50](answerbook/vector@567de50) - GitHub * add support for YAML and JSON to the generate command (vectordotdev#18345) [e15aec7](answerbook/vector@e15aec7) - GitHub * **appsignal sink**: Normalize metrics (vectordotdev#18217) [61c0ae8](answerbook/vector@61c0ae8) - GitHub * Begin publishing armv7hl rpm packages (vectordotdev#18387) [4c901ed](answerbook/vector@4c901ed) - GitHub * change dedupe config paths to `ConfigTargetPath` (vectordotdev#18241) [4ec6c11](answerbook/vector@4ec6c11) - GitHub * **dev**: add `ENVIRONMENT_AUTOPULL` override to Makefile (vectordotdev#18446) [bc6c421](answerbook/vector@bc6c421) - GitHub * **dev**: add environment networking overrides to Makefile (vectordotdev#18654) [89697d1](answerbook/vector@89697d1) - GitHub * **dev**: add networking overrides to website Makefile (vectordotdev#18655) [75ebda0](answerbook/vector@75ebda0) - GitHub * **dev**: decouple syslog source and codec features (vectordotdev#18381) [aca3a29](answerbook/vector@aca3a29) - GitHub * disable vrl 'string_path' feature (vectordotdev#18188) [5ce5ff1](answerbook/vector@5ce5ff1) - GitHub * **exec source**: add support for customizing command environment (vectordotdev#18223) [e27684c](answerbook/vector@e27684c) - GitHub * **metrics**: support Datadog metric origin metadata (vectordotdev#18405) [587c2e7](answerbook/vector@587c2e7) - GitHub * **new sink**: add AWS Simple Notification Service `aws_sns` sink (vectordotdev#18141) [7b2bddc](answerbook/vector@7b2bddc) - GitHub * **playground**: Create built.rs with versions and expose versions to the UI (vectordotdev#18424) [ce7da4e](answerbook/vector@ce7da4e) - GitHub * **route transform**: Add option to enable/disable unmatched output (vectordotdev#18309) [71343bd](answerbook/vector@71343bd) - GitHub ### Miscellaneous * Merge pull request vectordotdev#390 from answerbook/feature/LOG-18931 [0ce8b8d](answerbook/vector@0ce8b8d) - GitHub [LOG-18931](https://logdna.atlassian.net/browse/LOG-18931) * Merge branch 'master' into feature/LOG-18931 [e8a7404](answerbook/vector@e8a7404) - Darin Spivey [LOG-18931](https://logdna.atlassian.net/browse/LOG-18931) * Merge tag 'v0.33.1' into feature/LOG-18931 [c5efad4](answerbook/vector@c5efad4) - Darin Spivey [LOG-18931](https://logdna.atlassian.net/browse/LOG-18931) [LOG-18931](https://logdna.atlassian.net/browse/LOG-18931) * updating the doc, 2 urls were 404 (vectordotdev#18949) [904cb67](answerbook/vector@904cb67) - Jesse Szwedko * [WEB-3464] Adds TrustArc cookie consent banner (vectordotdev#18741) [69b4623](answerbook/vector@69b4623) - Jesse Szwedko * enhancement (journald source): Add `extra_journalctl_args` option to specify arbitrary command line arguments to journalctl (vectordotdev#18568) [21aec0c](answerbook/vector@21aec0c) - GitHub * Bump `chrono` to 0.4.27 (vectordotdev#18436) [e9feabd](answerbook/vector@e9feabd) - GitHub * fix(codecs) csv encoding quoting bug (vectordotdev#18320) [93ad80d](answerbook/vector@93ad80d) - GitHub * 0.32.0.cue typo (vectordotdev#18270) [f124e70](answerbook/vector@f124e70) - GitHub * Add notes about `ingress_upstreaminfo` log format for `parse_nginx_log()` function (vectordotdev#18477) [5cfb3e4](answerbook/vector@5cfb3e4) - GitHub * add PGO information (vectordotdev#18369) [eac3cd0](answerbook/vector@eac3cd0) - GitHub * **aws provider**: Use FIPS endpoints when configured to do so (vectordotdev#18390) [02c1b4c](answerbook/vector@02c1b4c) - GitHub * **ci**: Add protobuf compatibility check (pt 1) (vectordotdev#18552) [ddb5195](answerbook/vector@ddb5195) - GitHub * **ci**: protobuf compatibility check (pt 2) (vectordotdev#18553) [1eb933d](answerbook/vector@1eb933d) - GitHub * **core**: Add CLI arg and env variable to control openssl probing (vectordotdev#18229) [a4d73ca](answerbook/vector@a4d73ca) - GitHub * **core**: default tokio worker threads to `std::thread::available_parallelism()` (vectordotdev#18541) [730bb15](answerbook/vector@730bb15) - GitHub * **deps**: remove openssl legacy provider flag and update docs (vectordotdev#18609) [a6b1bed](answerbook/vector@a6b1bed) - GitHub * Editorial edits for updated component descriptions (vectordotdev#18590) [d179c57](answerbook/vector@d179c57) - GitHub * **enterprise**: configurable app name (vectordotdev#18554) [2e7e7e7](answerbook/vector@2e7e7e7) - GitHub * **es sink**: separate aws support in es & prometheus sink (vectordotdev#18288) [e652ea4](answerbook/vector@e652ea4) - GitHub * **file source**: fix some typos (vectordotdev#18401) [a3a1ef0](answerbook/vector@a3a1ef0) - GitHub * Fixed NixOS page (vectordotdev#18396) [89f0f08](answerbook/vector@89f0f08) - GitHub * **http_server source**: Configurable http response code (vectordotdev#18208) [afdc66e](answerbook/vector@afdc66e) - GitHub * **kubernetes_logs source**: Expose `oldest_first` (vectordotdev#18376) [dc66566](answerbook/vector@dc66566) - GitHub * **tls**: add new dedicated page for TLS configuration (vectordotdev#18844) [9ca6c7b](answerbook/vector@9ca6c7b) - Jesse Szwedko * **websocket sink**: Allow any data type depending on configured codec (vectordotdev#18295) [1c303e8](answerbook/vector@1c303e8) - GitHub ### **BREAKING CHANGES** * **config:** fix concurrency default & docs (vectordotdev#18651) * **deps:** remove openssl legacy provider flag and update docs (vectordotdev#18609) * **datadog_logs sink:** Use `endpoint` config setting consistent with the other datadog_ sinks. (vectordotdev#18497)
Overview
Adds support to Vector for handling Datadog metric origin metadata.
datadog_agentsource, or from an upstream Vector instance.Local Testing
emit counter and distribution from dogstatsd client into the Agent, receive in Vector with
datadog_agentsource, sent to Datadog withdatadog_metricssink. Validated that the pass through origin values from the Agent are being honored. Note that currently the Agent is not enriching sketches with Origin metadata, so Vector sends nothing in this case, as expected.at compile time set the
ORIGIN_PRODUCTenv var, validate this overrides the default when Vector is the metric's originator.use
host_metricsto validate correct origin metadata values when Vector is the metric's originator.NOTE
This changes the Vector protobuf definition to add a more complete structure for passing through the
EventMetadata.Also notably, we seem to have had a unit test that has been passing due to data files being committed statically to the repo, that haven't been kept up over time. The
native_jsoncodec is not fully compatible with thenativecodec- theEventMetadatais not being serialized/deserialized at all in the JSON case , but in the proto case, it is.In this PR I've simply commented out the failing test case, since that is already broken functionality on master.